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PREFACE 


This  document  was  prepared  for  the  Institute  for  Com- 
puter Sciences  and  Technology,  National  Bureau  of  Standards, 
by  the  Federal  Computer  Performance  Evaluation  and  Simula- 
tion Center  (FEDSIM).  It  is  based  on  a  similar  document 
originally  prepared  for  use  by  the  U.S.  Air  Force  in  support 
of  military  analyses.  That  document  has  been  rearranged, 
examples  have  been  changed,  and  the  document  has  been  made 
more  generally  applicable  so  that  it  may  be  used  throughout 
the  Federal  simulation  community.  Recommendations  for  im- 
provements of  these  guidelines  are  solicited.  It  is  intend- 
ed that  after  thorough  reviews  and  trial  use  this  document 
will  be  reissued  as  a  Federal  Guideline  in  the  series  of 
Federal  Information  Processing  Standards  (FIPS).  All  com- 
ments should  be  directed  to: 


Institute   for  Computer   Sciences  and  Technology 

Programming   Science  Division 
Washington,   D.    C.  20234 
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COMPUTER  MODEL  DOCUMENTATION  GUIDE 


This  document  provides  guidelines  for  prepar- 
ing documentation  for  computer  models.  Recommend- 
ed structures  for  four  types  of  manuals  providing 
model  information  for  four  different  classes  of 
audiences  (managers,  users,  analysts,  and  program- 
mers) is  presented.  This  document  specifies  the 
content  of  sections  and  subsections  for  each  type 
of  manual.  Manuals  prepared  using  these  guide- 
lines will  enable  persons  interested  in  a  model  to 
understand  the  capabilities  and  limitations  of 
that  model. 

Key  words:       documentation;         manuals;  models; 
8  imulation . 

I.  INTRODUCTION 

This  draft  document  provides  guidelines  for  preparing 
documentation  for  computer  models,  as  well  as  complete  sub- 
models delivered  separately.  The  primary  goal  of  model  do- 
cumentation is  to  communicate  effectively  the  details  of 
model  design  and  operation  to  persons  with  varying  interests 
in  a  model.  Since  a  model's  developers  are  frequently  not 
the  model's  ultimate  users,  complete,  concise  documentation 
is  essential  for  effective  model  use.  Documentation  should 
inform  analysts  familiar  with  the  phenomena  being  modeled, 
or  the  modeling  techniques  employed,  of  the  essential 
features  and  assumptions  of  a  new  model.  Throughout  its 
life  cycle,  a  model  may  be  used  and  modified  by  various  peo- 
ple, making  accurate  and  current  documentation  of  the  under- 
lying computer  program  essential  for  proper,  correct  use  and 
maintenance  of  the  model.  Ultimately,  model  results  may  be 
used  in  a  decision-making  environment  by  individuals  who  are 
unfamiliar  with  the  details  of  modeling  and  the  associated 
benefits,  risks,  and  costs.  In  such  situations,  model  docu- 
mentation should  describe,  in  non- technical  terms,  the  en- 
vironment in  which  a  model  can  be  useful,  limitations  on  its 
use,  and  the  manpower,  time,  and  dollar  costs  required  by 
its  use.  These  guidelines  recommend  structures  and  some 
conventions  for  preparing  model  documentation  in  the  form  of 
manuals  for  users,  analysts,  programmers,  and  managers. 
Each  type  of  manual  should  provide  clear,  concise  documenta- 
tion that  is  directed  toward  an  audience  with  a  particular 
interest   in  a  model. 
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These  guidelines  devote  a  section  to  each  type  of  manu- 
al, i.e..  The  Manager's  Manual,  The  User's  Manual,  The 
Programmer's  Manual,  The  Analyst's  Manual.  Each  section  be- 
gins with  a  table  of  contents  that  lists  recommended  topics 
of  interest  to  users  of  that  manual.  Items  may  be  added  to 
or  deleted  from  this  table  of  contents,  however,  according 
to  individual  requirements.  The  discussion  for  each  manual 
enlarges  upon  the  items  required  in  each  of  the  sections  and 
subsections,  as  recommended  in  the  table  of  contents  for 
that  manual.  Terms  used  in  each  manual  should  be  those 
directed   toward   that  manual's  audience. 


These  guidelines  are  for  models  that  are  used  chiefly 
in  a  decision-making  environment.  Thus,  the  main  goal  of  a 
Manager's  Manual  is  to  assist  managers  to  make  decisions. 
To  accomplish  this,  the  Manual  must  describe  the  model  and 
its  application  to  managers  (including  the  management  that 
sponsored  the  model)  who  may  be  interested  in  using  a 
developed  capability.  The  Manual  should  provide  managers 
with  sufficient  information  to  permit  them  to  accurately  as- 
sess model  input  requirements  (including  time,  money,  and 
other  resources),  available  outputs,  and  the  accuracy  and 
precision  of  the  results.  Managers  can  use  this  Manual  in 
justifying  the  employment  of  the  model  and  in  evaluating 
subsequent  results. 

A  user  is  assumed  to  be  interested  mainly  in  deriving 
results  from  a  model  for  specific  applications.  The  guide- 
lines recommend  that  the  User's  Manual  be  organized  into  a 
section  for  the  user  and  a  section  for  the  data  technicians 
who  will  set-up  and  run  the  model.  To  use  the  model  intel- 
ligently, a  user  must  be  aware  of  its  logical  structure,  the 
general  simulation  approach,  and  any  assumptions  and  limita- 
tions affecting  the  model's  applicability.  A  user  need  not 
be  interested  in  details  of  programming  or  analysis  beyond 
the  preparation  of  input  data  and  the  interpretation  of 
model  results. 

Programmers  are  interested  primarily  in  maintaining  and 
modifying  a  model.  A  programmer  must  correct  any  errors 
discovered  during  model  usage  that  are  not  attributable  to 
user-entered  data.  Programmers,  especially  those  required 
to  convert  a  model  to  another  computer  system,  need  to 
understand  features  of  a  model  that  are  installation  unique. 
Thus,  the  Programmer's  Manual  must  provide  all  the  details 
necessary  to  understand  the  operation  of  a  model:  to  debug 
it,  to  maintain  and  modify  it,  and  to  convert  the  model  to 
other  computer  systems. 
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These  guidelines  assume  an  analyst  to  be  Interested 
primarily  In  the  analytical  techniques  and  algorithms  used 
in  a  model.  An  analyst  Is  concerned  with  the  equations  used 
in  a  model  and  the  methods  used  for  model  verification  and 
validation.  An  analyst  does  not  need  to  know  user  details 
such  as  input  and  output  formats,  or  programming  details  in- 
volving  language  syntax. 

Decisions  about  which  of  these  manuals  are  actually  re- 
quired, whether  or  not  they  should  be  prepared  in  separate 
volumes,  etc.,  should  be  made  on  a  case-by-case  basis* 
Also,  a  plan  should  be  developed  for  documentation  updates 
and  maintenance,  so  that  these  manuals  remain  current.  Such 
issues  as  these  should  be  dealt  with  early  during  the  model 
planning  and  development  phases,  so  that  documentation  re- 
quirements actually  become  part  of  the  development  plan, 
rather  than  an  afterthought.  Further,  applicable  documenta- 
tion produced  using  programming  conventions  should  be  used 
in  conjunction  with  these  guidelines. 

Other  guidelines  prepared  specifically  to  support  com- 
puter software  documentation  are  available  which  may  be  used 
in  conjunction  with  this  guideline.     These   documents  are: 

FIPS  PUB  30,  Software  Summary  for  Describing  Computer  Pro- 
grams and  Automated  Data  Systems.  It  is  used  to  announce 
computer  programs  which  are  transferable,  and  have  broad  ap- 
plicability. A  standard  software  summary  form  is  defined 
(SF-185),  which  permits  description  of  the  program  for  iden- 
tification, reference  and  dissemination.  This  form  is  used 
by  the  General  Services  Administration  for  registry  of  pro- 
grams and  for  publication  of  program  abstracts  in  the 
Federal   Software  Exchange  Catalog. 

FIPS  PUB  38,  Guidelines  for  Documentation  of  Computer  Pro- 
grams and  Automated  Data  Systems.  It  provides  guidance  to 
documentation  content  for  the  development  phase,  including 
requirements  documentation,  system  specifications,  user, 
operations  and  maintenance  manuals,   and   test  documentation. 

FIPS  PUB  64,  Guidelines  for  Documentation  of  Computer  Pro- 
grams and  Automated  Data  Systems  for  the  Initiation  Phase. 
This  document  provides  guidance  for  project  requests,  feasi- 
bility studies,   and  cost  benefit  analyses. 
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II.      GUIDELINES   FOR   PREPARING  A  MANAGER'S  MANUAL 


This  section  provides  a  recommended  structure  for  the 
Manager's  Manual  and  describes  the  contents  of  each  section 
and  subsection.  A  manual  prepared  using  these  guidelines 
will  provide  descriptions  of  model  capabilities  and  require- 
ments such  that  model  strengths  and  limitations  will  be  com- 
municated to  decision  makers  and  potential  users.  The 
Manager's  Manual  will  provide  managers  with  sufficient  In- 
formation to  permit  them  to  accurately  assess  model  Input 
requirements  (Including  time,  money,  and  other  resources), 
available  outputs,  and  the  accuracy  and  precision  of  the 
results.  Managers  can  use  the  Manager's  Manual  In  justify- 
ing the  employment  of  the  model  and  In  evaluating  subsequent 
results.  Figure  II-l  Is  a  recommended  table  of  contents  for 
preparing  a  Manager's  Manual.  The  sections  and  subsections 
Included  In  that  figure  list  suggested  topics  that  are  of 
Interest  to  managers.  Items  may  be  added  to  or  deleted  from 
this  table  of  contents,  however,  according  to  Individual  re- 
quirements . 

1.  Introduction 

The  Introduction  should  Identify  the  sponsoring  organi- 
zation, provide  the  background  of  the  project,  state  the 
purpose  of  the  model,  and  present  an  overview  of  the  remain- 
ing sections  in  the  manual.  A  common  Introduction  used  for 
other  manuals  prepared  for  a  model  may  be  used  only  if  that 
introduction  is  void  of  specialized  terms.  The  specific 
purpose  of  the  Manager's  Manual  should  be  Included  in  the 
introduction  in  a  statement  of   the  form: 

"The  purpose  of  this  manual  is  to  communicate  to 
management  the  capabilities  and  limitations  of  (model 
name ) . " 

2.  Model  Description 

This  section  should  provide  a  summary  of  model  capabil- 
ities and  limitations.  Use  high-level  block  diagrams  to 
clarify  the  narrative,   as  needed. 
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2.1  Capabilities 


This  subsection  should  briefly  summarize  the  capabili- 
ties of  the  model.  Include  highlights  of  mathematical  and 
engineering  concepts  (but  not  equations)  used  as  the  basis 
of  the  model.  Include  a  statement  of  the  model's  primary 
purpose.  For  example,  "the  model  can  be  used  to  determine 
the  daily  number  of  machines  in  a  job  shop  required  to  pro- 
cess the  daily  orders."  Provide  an  overview  of  functional 
details  that  explains  how  the  model  accomplishes  its  stated 
purpose.  Discuss  the  general  areas  of  the  model's  applica- 
bility, including  the  decision  making  environments.  For  ex- 
ample, describe  the  types  of  systems  and  situations  that  can 
be  simulated  by  the  model  (possibly  with  minor  changes),  in- 
cluding the  number  and  kinds  of  subsystems  that  can  be  simu- 
lated. For  example,  if  a  job  shop  model  includes  order  pro- 
cessing, machine  repair,  or  distribution  subsystems,  then 
their  descriptions  should  be  provided.  Also  include  the  re- 
lationship of  this  model  to  any  other  models  (i.e.,  another 
model  may  prepare   input   data   for   this  model). 

2.2  Input/Output  Classes 

Provide  a  short  discussion  on  the  different  classes  of 
input  data  required  to  drive  the  model  and  of  output  data 
generated  by  the  model.  For  example,  a  job  shop  model  might 
require  entering  the  number  of  production  centers,  the 
number  of  machines  per  production  center,  the  service  rates 
of  the  machines,  and  the  routing  of  the  jobs  (orders).  Ex- 
amples of  model  output  include  statistics  that  show  the 
utilizations  (percent  busy  time)  of  the  production  centers 
and  the  job  turnaround  (total  processing)  times.  Identify 
any  special  preprocessing  required  for  input  data,  as  well 
as  all  post-  processing  required   on  model  results. 

2.3  Assumptions  and  Limitations 

List  assumptions  and  limitations  concerning  the  appli- 
cability of  the  model.  Identify  any  restrictions  on  model 
usage  caused  by  accuracy  limitations  of  input  data  and  out- 
put quantities.  Provide  comments  on  levels  of  detail  in  the 
model  that  affect  the  model's  applicability.  For  example, 
an  analytical  representation  rather  than  a  detailed  simula- 
tion of  a  system  component  could  affect  model  application. 
Also  describe  any  use  of  random  parameters  that  may  affect 
the  accuracy  and  use   of  model  output. 
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3.     Model  Development  and  Experimentation 


This  section  should  describe  significant  model  experi- 
ments already  run  and  should  provide  details  on  model  verif- 
ication and  validation  procedures  used.  Include  Information 
on  the  model's  development  history  resource  costs  and  re- 
quirements,  and  use. 

3.1  Development  History 

This  subsection  should  provide  pertinent  details  of  the 
history  of  model  development.  Include  comments  on  any  al- 
ternative methods  to  computer  simulation  that  were  con- 
sidered. Provide  information  on  any  "lessons  learned"  dur- 
ing model  development,  such  as  cost  overruns,  model  develop- 
ment delays,  user  dissatisfaction  with  model  results,  insuf- 
ficient workload  data  to  support  current  and  future  model 
applications,  inadequate  model  documentation,  poorly  defined 
problems ,   e  tc . 

3.2  Verification/Validation 

This  subsection  should  describe  any  verification  and 
validation  procedures  performed  on  the  model.  Include  any 
analyses  performed  on  the  sensitivity  of  model  output  data 
to   variations   in  model   input  data. 

3.3  Model  Experiments 

Describe  significant  model  experiments  performed  and 
their  results.  Briefly  describe  the  purpose  of  each  experi- 
ment and  the  extent  to  which  each  experiment's  goals  were 
realized.  Discuss  the  management  decisions  affected  by  each 
experiment.  Discussion  of  major  model  experiments  may  be 
included   in  separate   subsections   (e.g.,    3.3.1,  3.3.2). 

3.4  Costs  and   Resource  Requirements 

This  section  should  provide  details  on  the  costs  and 
resource  requirements  of  the  model.  Include  the  cost  (in 
time  and  money)  of  collecting  and  validating  input  data. 
For  example,  long  and  costly  data  collection  efforts  may  be 
necessary.  Provide  comments  on  model  maintenance  and  exper- 
iment costs.  Discuss  job  turnaround  times  (including  typi- 
cal run  times)  and  peculiar  model  requirements  such  as  ab- 
normally large  core  requirements  or  long  run  times.  Include 
comments  on  model  portability  and  security  requirements,  as 
needed . 


4,      Current  and  Additional  Applications 

This  section  should  summarize  benefits  already  derived 
from  the  model  and  recommend  other  applications  for  the 
model. 

A  .  1     Current  Use 

This  subsection  should  briefly  describe  how  the  model 
has  been  used  by  management  in  its  decision-making  process. 
Provide  details  of  recommendations  and  conclusions  derived 
using   the  model. 

4,2     Additional  Applications 

This  subsection  should  provide  details  of  any  addition- 
al applications  and  uses  of  the  model  beyond  the  current 
usage.  Discuss  in  general  terms  any  extensions  and  enhance- 
ments to  the  model  which  are  feasible  and  could  improve  its 
utility.  Identify  any  extensions  which  have  been  scheduled 
or   planned . 

APPENDICES 

Two  appendices  should  be  provided  as  required.  Appen- 
dix A  should  reference  all  other  project  documentation  (in- 
cluding the  User's  Manual,  Analyst's  Manual,  and 
Programmer's  Manual),  including  references  to  the  organiza- 
tion and  person  responsible  for  maintaining  the  document. 
Include  references  to  any  documentation  of  experiments  per- 
formed using  the  model.  Appendix  B  should  list  all  applica- 
ble documents  (excluding  project  documentation  previously 
included  in  Appendix  A),  including  cited  and  uncited  refer- 
ences. 
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III.      GUIDELINES   FOR   PREPARING   A   USER'S  MANUAL 


This  section  presents  a  recommended  User's  Manual  or- 
ganizational structure  and  discusses  the  contents  of  sec- 
tions and  subsections  to  be  included  therein.  A  User's 
Manual  prepared  using  these  guidelines  will  enable  a  nonpro- 
gramming  model  user  to  understand  the  model's  logical  struc- 
ture, the  input  data  requirements,  the  results  produced  by 
the  model,  and  the  use  of  model  results.  Figure  III-l 
presents  a  recommended  table  of  contents  for  a  User's  Manu- 
al. The  sections  and  subsections  contained  in  the  figure 
cover  the  general  needs  of  a  user  interested  in  a  model.  In 
documenting  a  particular  model,  however,  sections  and  sub- 
sections may  be  added  to  improve  clarity,  and  some  subsec- 
tions may  be  omitted  for  simple  models.  Note  that  there  is 
a  certain  amount  of  redundance  among  the  various  sections  of 
a  User's  Manual  prepared  according  to  these  guidelines. 
Nevertheless,  the  progressively  increasing  level  of  detail 
dictated  by  this  structure  is  desirable  to  satisfy  different 
levels  of  user   interest   in  the  manual. 


1.  Introduction 


The  User's  Manual  introduction  should  contain  the  back- 
ground of  the  project,  the  purpose  of  the  model,  and  an 
overview  of  the  remaining  sections  in  the  manual.  A  common 
introduction  may  be  used  for  all  the  manuals  prepared  for  a 
model,  but  the  specific  purpose  of  the  User's  Manual  should 
be   included   in  a  statement  of   the  form: 

"The  purpose  of  this  manual  is  to  provide  nonprogram- 
ming  users  of  (model  name)  with  the  information  neces- 
sary to  use   the  model  effectively." 

2.     Description  of   the  Model 

This  section  should  contain  a  well-structured  presenta- 
tion of  the  logical  details  of  the  model.  The  material  here 
should  be  descriptive  and  include  block  diagrams  and  tables 
and  charts  where  needed;  it  should  not  give  details  needed 
by  a  data   technician  to  run   the  model. 

2 . 1  Overview 


This  subsection  should  provide  sufficient  general  in- 
formation about  the  model  to  assist  a  user  in  determining 
the  applicability  of   the  model   for   specific  needs. 
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2.1.1  Model  Identification  The  Identification  should  contain 
the  name  of  the  physical  system  being  simulated,  name  of  the 
model  (acronym  and  expansion),  programming  language(s)  used 
to  Implement  the  model,  computer(s)  on  which  the  model  may 
be  run,   and   relationships.   If  any,    to  other  models. 

2.1.2  Physical  System  Highlights.  Include  a  block  diagram 
that  shows  the  physical  system  or  phenomenon  being  simulat- 
ed. Discuss,  at  a  macro  level,  the  major  system  elements 
shown  In  the  diagram,  their  relation  to  each  other,  and  the 
flow  of  control,  information,  data,  and  activity  between 
them,  as  appropriate.  In  the  case  of  complex  models,  pro- 
vide in  this  subsection  a  first-level  block  diagram  that 
shows  the  major  subsystems  and  their  interactions,  and  post- 
pone the  details  of  each  of  the  subsystems  until  Subsection 
2.2.1,  Figure  III-2  is  an  example  of  physical  system 
highlights  depicting   the  operations  of  a   typical   shop  model, 

2.1.3  Mode  1  Applicability .  Discuss  the  general  magnitude  of 
model  applicability.  The  types  of  systems  or  situations 
that  can  be  simulated  by  the  model  (possibly  with  minor 
changes)  and  the  number  of  subsystems  (e.g.,  production 
centers  and  machines  per  production  center  in  a  job  shop 
simulation)  that  can  be  handled  are  examples  of  material  to 
be   included   in  this  subsection. 

2.1.4  Input  and  Output .  Provide  a  general  statement  of  the 
different  kinds  of  input  data  needed  to  drive  the  model,  the 
output  data  generated  by  the  model,  and  uses  of  model  out- 
put. For  example,  a  job  shop  model  might  require  entering 
the  number  of  production  centers,  the  number  of  machines  per 
production  center,  the  service  rates  of  the  machines,  and 
the  routing  of  the  jobs  (orders).  Examples  of  model  output 
include  statistics  that  show  the  utilizations  (percent  busy 
time)  of  the  production  centers  and  the  job  turnaround  (to- 
tal processing)  times.  The  principal  model  use  could  be  to 
determine  the  number  of  machines  in  a  job  shop  required  to 
process  the  daily  orders. 

Highlight  any  special  data  collection  procedures  (e.g., 
run  other  models  or  computer  programs,  extract  data  from  do- 
cuments or  listings,  conduct  sampling  experiments)  required 
to  produce  model  input  data.  List  any  unique  data  sources 
or  other  organizations  that  might  have  to  be  contacted  to 
gather  data.  Figure  III-3  is  an  example  of  an  input/output 
schematic . 
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2.2  Methodology 

This   subsection   should   provide   the   user  with  a  detailed 
understanding   of   how  the   model  works. 

2.2.1  Physical  System  Details .  This  subsection  is  an  ela- 
boration of  Subsection  2.1.2.  The  operations  that  take 
place  in  each  of  the  blocks  in  the  Physical  System  Block  Di- 
agram should  be  discussed  in  detail.  Detailed  block  di- 
agrams of  subsystems  should  be  provided  for  a  complex  model. 
The  level  of  detail  used  in  the  simulation  (e.g.,  the 
smallest  meaningful  time  increment  for  event-type  models, 
the  way  in  which  complex  system  interactions  are  simplified 
in   the   model)    should  be   clearly  indicated. 

2.2.2  Model  Logic  and  Data  Flow.  This  subsection  should 
describe  the  logical  flow  of  data  through  the  model,  from 
the  entry  of  input  data  to  the  generation  of  output  data. 
Include  a  schematic  that  indicates  the  major  model  software 
elements,  the  data  flow  between  model  elements,  and  model 
inputs  and  outputs.  Figures  III-4  and  III-5  are  two  types 
of  schematics  for  the  same  model.  Either  of  these  two 
types,  or  any  other  type  of  schematic  that  clearly  depicts 
model  logic  and  data  flow,  may  be  used.  Accompanying  dis- 
cussion should  relate  model  elements  and  data  flow  to  physi- 
cal system  elements  and  data  flow  described  in  Section 
2.1.2.  For  complex  models,  include  a  table  that  relates 
physical  system  names  to  the  program  segments  that  simulate 
them. 

2.3  Assumptions   and  Limitations 

All  the  system-related  assumptions,  assumptions  on 
model  parameters  (e.g.,  hard-coded  values),  limitations  on 
output  accuracy,  and  any  restrictions  on  the  use  of  the 
model   should   be   discussed   in  detail. 


2.3.1  Sy s  tem-Re lated  As  sumpt  ions  and  Limltat  ions  »  List  any 
assumptions  that  limit  or  describe  the  kinds  of  systems  or 
phenomena  that  are  treated  in  the  model.  For  example,  a 
description  of  a  job  shop  model  should  define  the  model's 
boundaries  (i.e.,  the  subsystems  that  the  model  includes), 
the  kinds  of  activities  simulated  (e.g.,  machine  failures 
and   repairs),  etc. 

2.3.2  Model  Parameters .  List  the  valid  ranges  for  principal 
model  input  parameters  (e.g.,  the  maximum  and  minimum 
number  of  subsystems).  Also  list  values  for  any  parameters 
that  are  included  in  the  model  software  and  cannot  be  modi- 
fied by   the   use  r  . 
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2,3v3  Output  Limitations .  List  any  limitations  on  output 
usage  caused  by  Inaccuracy  of  the  output  data.  For  example, 
all  digits  in  an  output  data  field  may  not  be  significant 
because  the  input  data  are  estimates,  and  high  precision  in 
the  output  data  is  either  unobtainable  or  inappropriate. 

2.3.4  Restrictions  on  Mode  1  Use .  Enumerate  all  restrictions 
on  model  usage.  For  example,  a  job  shop  model  restriction 
might  be  that  only  flrst-in,  first-out  queuing  disciplines 
are  modeled  for  the  production  centers. 

3.     Model   Input  Data 

This  section  should  describe  in  detail  all  the  input 
data  needed  to  run  the  model.  The  material  in  this  and  the 
four  subsequent  sections  should  serve  as  a  reference  for 
both  the  user  and   the   data   technician  who  runs  the  model. 

3.1  General  Description 

This  subsection  should  describe  the  overall  input  data 
structure  and  the  data  media  (tape,  cards,  disk  data  sets, 
etc.).  Include  a  table  that  shows  input  data  set  names, 
their  media,  and  any  general  data  limitations.  Also, 
describe  the  Interdependence,  if  any,  of  input  data  sets. 
(Detailed  descriptions  of  Individual  data  items  within  the 
input  data  sets   should  be   left   for  Section  3.2.) 

3.2  Detailed  Descriptions 

Input  data  items  are  normally  organized  In  related 
groups,  such  as  machine  performance  characteristics,  job 
processing  requirements,  etc.,  or  as  the  data  items  that  are 
entered  on  one  punch  card.  These  related  groups  of  data  es- 
tablish and  define  a  data  set  and  should  be  described  to- 
gether. The  input  data  sets  and  the  items  within  each  data 
set  should  be  discussed  in  the  order  of  their  appearance  in 
the  run  stream.  For  each  input  data  set,  provide  the  fol- 
lowing Information  (each  data  set  description  should  begin 
on  a  new  page). 

3.2.1  Data  Set  Name . 

In  this  subsection,  give  an  overview  of  the  data  set's  con- 
tents and   its  purpose. 

3.2.1.1  Numbe r  o f  Inputs .  Indicate  the  number  of  data  sets 
of  this  type  and  the  maximum  number  of  data  items  in  the 
data  set  that  may  (or  must)  be  used  in  the  simulation.  Dis- 
cuss any  factors  that  influence  the  total  number  of  inputs 
from  this  data  set. 
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3.2.1.2  Other  Re lated  Data  Sets.  List  any  data  sets  whose 
contents  depend  on  or  dictate  the  input  values  for  this  data 
set.  Discuss  the  relationships  between  data  items  In  the 
data  sets. 

3.2.1.3  Description  of  Data  I  terns .  In  this  subsection,  pro- 
vide general  comments  on  the  format  of  the  data  items  (e.g., 
free  form.  Integer  in  card  columns  8-11,  NAMELIST)  followed 
by  the  description  of  each  of  the  data  items.  For  each 
item,  the  following  should  be  given:  name,  type,  format  (if 
fixed),  permissible  range  or  fixed  value,  unit  of  measure- 
ment, default  value  (value  assumed  by  the  program  when  the 
item  is  omitted),  definition  of  the  item  describing  how  it 
is  used  in  the  model,  relationship  to  other  data  items. 
Tables   should  be  used  where  appropriate. 

3.2.1.4  Sample  Input . 

A  format  layout  should  be  provided  for  the  data  set  to  pro- 
vide the  user  a  visual  reference  for  preparing  the  input 
data . 

3.3     Data  Collection  and  Maintenance 

An  Important  part  of  model  application  is  data  collec- 
tion. Therefore,  it  is  necessary  to  include  appropriate  in- 
structions on  data  collection  and  maintenance.  Specific 
responsibilities  need  to  be  assigned  to  analysts  and  users 
for   these  functions. 

3.3.1  Data  Sources .  Discuss  the  data  sources  for  each  input 
data  set.  The  discussion  should  identify  the  form  in  which 
raw  data  are  available,  other  organizational  elements  from 
which  the  data  must  be  collected,  if  appropriate,  and  the 
time  required   to  collect   the  data. 

3.3.2  Collect  ion  Procedures .  Describe  any  special  statisti- 
cal techniques  or  experiments  for  obtaining  the  data.  Iden- 
tify any  other  computer  programs  or  models  that  must  be  used 
to  collect  or  process  data,  and  list  or  reference  instruc- 
tions for  their  use.  Where  appropriate.  Include  a  flowchart 
that  illustrates  the  major  data  collection  steps  and  their 
sequence.  Figure  III-6  is  an  example  of  special  procedures 
to  be  used  in  obtaining  data  for  a  model.  In  this  example, 
the  type  and  frequency  of  orders  are  analyzed  along  with 
production  center  performance  data  to  produce  a  statistical 
data  base.     This,  data  base   is   then  used  as   input   to  a  model 
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of  the  order  handling  process.  The  order  model  produces 
data  that  profiles  the  arrival  patterns,  routing  distribu- 
tions, and  processing  requirements  of  the  jobs.  These  data 
are.   In  turn,   used   as  Input   to   the   job  shop  model. 

3.3.3  Updating  Procedures .  Give  step-by-step  procedures  for 
maintaining  the  data  sets  and  preparing  them  for  new  experi- 
ments. Identify  any  other  computer  programs  that  must  be 
used  to  update  the  data  sets,  and  list  or  reference  Instruc- 
tions for  their  use.  Where  appropriate.  Include  a  flowchart 
that  Illustrates  the  major  update  procedures  and  their  se- 
quence. 

A.     Model  Output  Data 

This  section  should  describe  In  detail  all  the  output 
data  produced  by  the  model  and  should  Indicate  their  mean- 
ings and  uses. 

4.1     General  Description 


Discuss  the  overall  output  structure  In  this  section. 
Indicate  the  number  and  types  of  output  data  sets,  output 
media,  correlation  between  outputs,  quantity  of  output  (op- 
tional and  mandatory),  and  postprocessing,  If  any,  that 
should  be   performed  on  the   output  data. 

4.2     Detailed  Description 

For  each  output  data  set  (or  major  group  of  logically  con- 
nected data  Items),  provide  the  following  Information  (each 
output   data   set   should  begin  on  a  new  page). 

4.2.1  Data  Set  Name .  Give  the  full  name  or  acronym  of  the 
output  data  set  or  group  of  data  under  this  subheading. 
Give  an  overview  of  the  data  set's  contents,  its  purpose, 
and   its   relation  to  other  model  results. 

4.2.1.1  De  script  ion  of  Items.  Each  output  item  should  be 
Included  in  a  table  that  shows  its  name,  a  brief  descrip- 
tion, and  gives  information  to  use  in  validity  checking,  if 
appropriate.  Accompanying  discussion  should  expand  on  each 
Item's  description  and  should  show  how  the  items  are  derived 
or  calculated.  Include  mathematical  formulae  where  ap- 
propriate . 

4.2.1.2  Interpre  tat  ion .  Explain  how  the  data  items  can  be 
used,  and  describe  actions  to  be  taken  for  any  subsequent 
runs  based  on  the  output. 
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^.2.1.3  Sample  Output .  Include  a  sample  of  the  output  data 
set.  A  sample  format  is  satisfactory  where  it  is  not  prac- 
tical  to  provide  an  actual  sample. 

5.  Run  Preparation  Instructions 

This  section  of  the  User's  Manual  should  describe  pro- 
cedures for  organizing  the  input  data  to  submit  computer 
runs  as  discussed   in  Section  3. 

5.1  Run-Stream  Description 

This  subsection  should  give  a  pictorial  (or  tabular) 
representation  of  the  deck  constituting  the  run-stream  that 
shows  all  the  control  cards  and  the  data  cards  in  proper  se- 
quence. Mandatory  and  optional  cards  should  be  discussed. 
If  the  model  is  interactive,  include  comments  on  any  special 
techniques  used   for   interactive   submission  of  jobs. 

5.2  Resource  Requirements 

This  subsection  should  describe  the  computer  resources 
required  by  the  model.  These  include  main  memory,  mass 
storage,  number  of  tape  units,  execution  time,  numbers  of 
punched  cards,  and  printed  lines  expected  as  output.  If  the 
computer  resources  vary  depending  on  input  data,  provide 
aids   to  estimate  them. 

5.3  Restart/Recovery  Procedures 

For  models  that  require  large  amounts  of  computer 
resources  it  is  important  to  recover  from  abnormal  termina- 
tions and  to  restart  the  job.  If  any  such  provisions  are 
made  in  the  model  design,  they  should  be  discussed  in  this 
subsect  ion . 

6.  Sample  Model  Run 

Include  a  sample  run  that  illustrates  the  complete  in- 
put scenario  and  the  resulting  output  to  assist  a  beginning 
user  in  making  a  test  run  and  verifying  correctness  of  pro- 
cedures . 
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7.     Trouble-Shooting  Guide 

Tabulate  user  input  error-messages  produced  by  the 
model  software,  and  describe  the  required  corrective  action. 
Since  other  errors  should  be  handled  by  programmers,  those 
errors   should   be  discussed   in   the   programmer's  manual. 

APPENDICES 

Three  appendices  should  be  provided  as  required.  Ap- 
pendix A  should  provide  an  alphabetical  listing  of  all  ab- 
breviations and  acronyms  used  in  the  User's  Manual.  Appen- 
dix B  should  list  all  specialized  User's  Manual  terms  and 
their  definitions.  All  applicable  documents,  including  cit- 
ed  and   uncited    references,    should   be   provided   in  Appendix  C. 
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IV.      GUIDELINES   FOR   PREPARING   A   PROGRAMMER'S  MANUAL 


This  section  provides  a  recommended  organization  for  a 
Programmer's  Manual  and  describes  the  contents  of  each  sec- 
tion and  subsection  proposed  for  that  manual.  A 
Programmer's  Manual  written  using  these  guidelines  will  en- 
able a  programmer  to  maintain  and  modify  a  model.  The 
guidelines  will  provide  all  the  details  necessary  for  a  pro- 
grammer to  understand  the  operation  of  the  model  and  to 
trace  through  it  for  debugging,  for  making  modifications, 
and  for  determining  if  and  how  the  model  can  be  converted  to 
other  computer  systems.  Figure  IV-1  is  a  recommended  table 
of  contents  for  a  Programmer's  Manual.  The  sections  and 
subsections  included  in  the  figure  cover  the  general  needs 
of  a  programmer  interested  in  a  model.  In  documenting  a 
particular  model,  however,  sections  and  subsections  may  be 
added  to  improve  clarity,  and  some  subsections  may  be  omit- 
ted for  simple  models.  Any  appropriate  documentation  pro- 
duced using  a  program  documentation  language  could  be  used 
to   satisfy  the   guidelines   contained  herein. 

1.  Introduction 

The  introduction  to  the  Programmer's  Manual  should  con- 
tain the  background  of  the  project,  the  purpose  of  the 
model,  and  an  overview  of  the  remaining  sections  in  the 
manual.  A  common  introduction  may  be  used  for  all  the  manu- 
als prepared  for  a  model,  but  the  specific  purpose  of  a 
Programmer's  Manual  should  be  included  in  a  statement  of  the 
form: 

"The  purpose  of  this  manual  is  to  provide  programmer 
personnel  of  (model  name)  with  the  information  neces- 
sary  to  effectively  maintain  and  modify   the  model." 

2.  Model  Specifications 

This  section  should  provide  a  summary  of  the  model's 
specifications,  including  capabilities  (i.e.,  problems  ad- 
dressed and  methods  of  solution),  a  description  of  the  host 
computer  system,  and  the  processing  requirements  (i.e., 
memory,  peripherals,  languages)  placed  by  the  model  on  that 
host  system.  The  details  should  be  presented  in  tabular 
form  (supplemented  by  narrative  description,  as  appropri- 
ate), whereby  one  table  describes  the  complete  modeling  sys- 
tem and  additional  tables  describe  major  submodels  or  pro- 
grams  as   needed   for  clarity. 


-23- 


1.  Introduction 

2.  Model  Specifications 

3.  Model  Description 

3.1  Processing 

3.1.1  Overview 

3.1.2  Major  Components 

3.1.3  Model  Initialization  and  Wrap-up 

p. 2    Data  Structures 

3.2.1     Local  Data  Structures 
*         3.2.2    Global  Data  Structures 
t         3.2.3    Special  Data  Structures 

3.3  Overlays 

3.4  Model  Modifications 

3.4.1  Planned  Maintenance 

3.4.2  Other  Changes 

4.  Description  of  Routines 

4.1     Routine  Name  (First  Routine) 


1. 

1 

Purpose 

1. 

2 

Type 

1. 

3 

Calling  Sequence 

1. 

4 

Argument  Definition 

1. 

5 

Calling  Routines 

1. 

6 

Called  Routines 

1. 

7 

Files 

1. 

8 

Error  Messages 

1. 

9 

Narrative 

1. 

10 

Block  Diagrams 

1. 

11 

Sample  Test  Run 

5.  Data  Base  Description 

5.1     File  Name  (First  File) 

5.1.1  Purpose 

5.1.2  Format 

5.1.3  Routines 

5.1.4  Updating 

6.  Source  Listing 

7.  Error  Messages 
APPENDICES 

A.  Glossary 

B.  Bibliography 

C.  Index 

D.  Model  Test  Results 


RECOMMENDED  TABLE   OF  CONTENTS   FOR  A 
PROGRAMMER'S  MANUAL 

FIGURE  IV-1 


-24- 


3.     Model  Description 


This  section  should  contain  a  we  1 1 - s t r uc t ured  presenta- 
tion with  emphasis  on  the  operational  details  of  the  model. 
The  discussion  should  be  written  in  an  easy-to-understand 
manner  that  cross-references  special  model  language  terms 
with  modeled  system  features  whenever  possible.  This  sec- 
tion should  be   divided   into   four  subsections. 


3 .  1  Processing 


This  subsection  should  provide  details  on  model  opera- 
tions for  programmers  who  need  to  understand  the  processing 
techniques  used  in  the  model.  The  discussion  should  be  at 
the  Macro  level,  with  a  discussion  of  internal  routine  de- 
tails postponed  until  Section  4  of  this  manual.  Details  on 
I/O  formats  and  default  input  data  values  should  be  reserved 
for  the  User's  Manual.  Block  diagrams  should  be  used  as 
necessary  to   supplement   the  narrative. 

3.1.1  Overview .  This  subsection  should  present,  in  modeled 
system  terminology,  an  overview  of  the  problem  solved  by  the 
model.  Include  a  discussion  of  the  basic  tasks  modeled. 
Figure  IV-2  is  an  example  of  a  block  diagram  that  could  sup- 
plement a  narrative  description   in  this  subsection. 

3.1.2  Ma  jor  Components .  This  subsection  should  describe  the 
flow  of  data  or  control  information  through  the  model  at  the 
major  routine,  or  routine  group,  level.  Include  detailed 
block  diagrams  that  depict  paths  among  the  modeled  tasks, 
highlighting  major  decision  points  in  the  logic  flow.  This 
subsection  may  contain  as  many  levels  of  discussion  as  are 
necessary  to  clearly  describe  model  operation. 

3.1.3  Model  Init  ializat  ion  and  Wr ap-Up .  This  subsection 
should  note  any  differences  in  the  performance  of  model 
tasks  accomplished  during  model  initialization  and  wrap-up 
and  those  same  tasks  when  performed  during  normal  process- 
ing . 

3 . 2  Data  Structures . 

This  subsection  should  provide  information  on  all  data 
structures  internal  to  the  model.  Include  descriptions  of 
local  and  global  variables,  arrays,  and  data  sets,  as  well 
as  any  special  data  structures,  such  as  the  set-entity  rela- 
tionships in  SIMSCRIPT.  If  required  for  understanding, 
separate  descriptions  of  each  array  index  should  be  provid- 
ed . 
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EXAMPLE  OF   A  MODEL  OVERVIEW  BLOCK  DIAGRAM 
FIGURE  IV-2 
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3.2.1  Local  Data  St ructure s .  This  subsection  should  contain 
the  meaning  and  purpose  of  all  local  variables,  arrays,  and 
data  sets  (local  data  structures  have  their  values  defined 
only  within  particular  routines).  To  improve  clarity,  local 
data  structures  should  be  associated  with  the  routines  in 
which  they  appear. 

3.2.2  Global  Data  St  ructures .  This  subsection  should  con- 
tain the  meaning  and  purpose  of  all  global  variables,  ar- 
rays, and  data  sets  (global  data  structures  are  defined 
throughout  the  model).  Include  an  alphabetized  list  of  glo- 
bal data  structures  (including  special  data  structures), 
cross-referenced  by  the  routines  in  which  they  appear,  and 
the  source  code  li  ne  numbers  (Table  IV— 1).  Source  code  line 
numbers  can  be  obtained  from  the  source  listing  in  Section  6 
of  this  manual.  Examples  of  global  data  structures  that 
should  be  Included  in  this  subsection  are  the  COMMON  blocks 
of  FORTRAN. 


3.2.3  Special  Data  St  ructure  s .  Any  special  data  structures, 
both  local  and  global,  should  be  listed  and  described  in 
this  subsection.  For  example,  a  job  shop  model  implemented 
in  SIMSCRIPT  might  represent  the  jobs  with  temporary  enti- 
ties, the  job  processing  requirements  with  entity  attri- 
butes, and  the  sequence  of  production  centers  required  to 
process  the  job  with  a  set  (owned  by  the  job  with  production 
centers  as  numbers).  A  GPSS  implementation,  however,  might 
represent  the  jobs  with  transactions,  the  processing  re- 
quirements with  transaction  parameters,  and  the  route  with  a 
row  in  a  matrix  save  value  (the  columns  contain  the  sequence 
of  production  centers). 

3.3  Overlays 

If  the  model  is  overlayed,  this  subsection  should  pro- 
vide details  of  the  overlay  design  decisions  that  determined 
the  overlay  strategy.  Included  should  be  a  narrative  and  a 
block  diagram  description  of  the  control  flow  of  the  over- 
lays and  their  interactions.  Figure  lV-3  contains  a  sample 
overlay  structure  with  a  main  program,  four  primary  overlay 
segments,  and  five  secondary  overlay  segments.  Routines 
residing  in  each  overlay,  and  their  memory  requirements, 
should  be  listed  (Table  IV-2).  References  should  be  made 
to  the  discussion  of  model  processing  in  this  manual  (Sub- 
section 3.1)   to  reinforce  or  clarify  the  overlay  discussion. 
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NAME  SOURCE  CODE 

NUMBER  ITEM  ROUTINE  LINE  NUMBER 


1  AMGO   (V)^         MAIN  15 

2  AOPS    (A)"^         FASTER  52 

3  BVBC   (V)  MAIN  12 

TFBAI  163 
PRNTREP  210 

212 
2  90 

4  CMCODE    (V)       MAIN  10 

ACSUMT  150 
PRNTREP  211 

286 

5  LEVDAT   (DS)     MAIN  13 

AUXSUM  12  0 

THBAB  184 
THBAI  195 
PRNTREP  250 

6  TCTV   (V)  MAIN  14 

ABFSTR  106 

108 

PRNTREP  210 

253 


Item  is  a  variable. 
Item  is  an  array. 
Item  is  a  data  set. 


CROSS-REFERENCED  DATA  STRUCTURE  LIST  SAMPLE 

TABLE  IV-1 
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OVERLAY 
SEGMENT  NUMBER 


ROUTINE 
NAME 


MEMORY 
REQUIREMENTS 
(KBYTES) 


1 

MAIN 

50 

ROUTl 

10 

ROUT  2 

20 

ROUTS 

20 

2 

PROCl 

100 

PR0C2 

25 

PR0C3 

25 

3 

COM  Pi 

75 

C0MP2 

50 

4 

SOLVl 

75 

S0LV2 

50 

5 

LSIGNl 

100 

• 

LSIGN2 

20 

LSIGN3 

15 

6 

COMPIO 

140 

7 

FINDl 

70 

FIND2 

48 

8 

SUMIO 

140 

9 

WRITER 

137 

10 

REPORT 

128 

SAMPLE  LIST  OF  ROUTINES  BY  OVERLAY  SEGMENT 

TABLE  IV-2 
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3. A     Model  Modifications 


This  subsection  should  include  information  concerning 
changes  in  model  software  and  data  bases.  Include  a 
description  of  any  programming  conventions  used  in  the  model 
(e.g.,  all  variables  referencing  one  data  base  may  begin 
with  a  specific  character).  In  addition,  this  subsection 
should  provide  procedures  needed  by  programmers  during  the 
model  compilation,  recompilation ,  and  execution  stages.  In- 
clude a  sample  control  card  setup  that  illustrates  each  of 
those  states,  including  mandatory  and  optional  cards.  If 
the  model  is  interactive,  include  comments  on  interactive 
procedures. 

3.4.1  Planned  Ma  intena nee .  This  subsection  should  identify 
all  planned  periodic  maintenance  on  the  model  and  its  data 
bases   (e.g.,    periodic   data   base  updates). 

3.4.2  Othe  r  Changes .  This  subsection  should  identify  pro- 
cedures for  making  all  modifications  to  the  model  other  than 
planned  periodic  maintenance.  Provide  details  for  making 
changes  necessitated  by  programming  errors  discovered  during 
model  usage,  as  well  as  changes  to  the  model  required  by 
changes  in  the  host  modeling  language.  Include  directions 
for  implementing  software  changes  to  produce  a  new  version 
of   the  model   (e.g.,    changes   in  model  applicability). 

A.      Description  of  Routines 

This  section  should  provide  a  detailed  description  of 
principal  model  routines.  Include  a  discussion  of  all  types 
of  routines  that  comprise  the  model  (i.e.,  event, 
subroutine,  function,  etc).  Provide  an  alphabetized  listing 
of  all  routine  names  along  with  calling  routines  and  called 
routines  (Table  IV-3)  or  a  block  diagram  showing  routine 
linkages,  as  needed.  Each  routine  should  be  described  in  a 
separate  subsection.  For  each  routine,  provide  the  follow- 
ing information. 

A.l     Routine   Name   (First  Routine) 

A. 1.1  Pur po se .  Briefly  state  the  purpose  of  the  routine 
(e.g.,  routine  ALLOCATE  computes  the  time  a  job  is  scheduled 
to   complete   its   processing   at   a   production  center). 

A. 1,2  Type .  Specify  the  type  of  routine  (i.e.,  function, 
subroutine).  A  description  of  all  routine  types  in  the 
model  should  be  contained  in  the  introductory  comments  of 
this   sect  ion . 
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^•1«3  Calling  Sequence .  List  all  variables,  arrays, 
pointers   in  the   routine   calling  sequence. 


4.1.4  Argument  De  f  ini t  ion  .     Define  all   routine  arguments. 

4.1.5  Calling  Rout  ine  s .  List  all  routines  that  call  this 
rout  ine . 

4.1.6  Called  Rout  ines .  List  all  routines  called  by  this 
routine . 

4.1.7  Files.     List  all  files  this  routine  creates  or  uses. 

4.1.8  Error  Me  s  sage  s .  Itemize  all  error  messages  which  can 
originate   in  this  routine. 

4.1.9  Narrative .  Include  a  narrative  description  as  neces- 
sary, to  amplify  and  highlight  subtleties  included  in  the 
code.  As  a  minimum,  include  any  equations  and  formulae 
referenced   from  the  Analyst's  Manual. 

4.1.10  Block  Diagrams .  Use  block  diagrams  or  other  documen- 
tation aids  (such  as  program  documentation  languages),  as 
required,    to  clearly  depict  operation  of   the  routine. 

4.1.11  Sample  Test  Run .  Provide  the  results  of  test  runs, 
along  with  values  of  input  data,  for  each  complex  routine  to 
assist   in  verifying  changes   to   those  routines. 

5.      Data  Base  Description 

This  section  should  discuss  all  mass  storage  files  used 
or  created  by  the  model.  Each  file  should  be  described  in  a 
separate  subsection  and  should  contain  the  following  infor- 
mation  (each  file  description  should  begin  on  a  new  page). 

5.1     File  Name   (First  File) 

Provide  the  full  name  or  acronym  of  all  the  model 
files. 

5.1.1  Pur po se .  Briefly  state  the  purpose  of  the  file  (e.g., 
contains   preprocessed   destination  data). 

5.1.2  Format .  Explain  the  format  of  the  file  (i.e.,  block, 
size,  record  size,  data  item  identification,  and  field 
sizes) . 
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NUMBER 


CALLING  ROUTINE 


CALLED  ROUTINE 


1 

ABFSTR 

NONE 

2 

ACSUMT 

TFBAI 

TKBAB 

AUXSuM 

ACSDMT 

4 

FASTER 

ABFSTR 

AUXSUM 

5 

MAIN 

FASTER 

6 

PRNTREP 

NONE 

7 

TFBAI 

PRNTREP 

8 

THBAB 

THBAI 

9 

THBAI 

PRNTREP 

CROSS-REFERENCED  ROUTINE  LIST  EXAMPLE 
TABLE  IV-3 
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5.1.3  Rout  1 ne s .  Identify  all  routines  that  use  or  create 
the  file. 

5.1.4  Updat ing .  Include  Instructions  for  file  maintenance 
and  updating  as  appropriate. 

6.  Source  Listing 

This  section  should  contain  the  source  code  of  the 
model.  If  the  source  listing  is  large,  it  should  be  bound 
separately  and  made  available  upon  request.  Also,  source 
listings  with  line  numbers  can  be  referenced  from  Subsection 
3.2.2  as  a  cross-reference   for  model  variables. 

7.  Error  Messages 

All  program-generated  error  messages,  the  names  of  the 
routines  in  which  they  are  generated,  and  suggested  correc- 
tive actions  should  be  listed  in  this  section.  Each  error 
message  may  be  described   in  a  separate  subsection. 

APPENDICES 

Four  appendices  to  this  manual  should  be  provided  as 
required.  Appendix  A  should  define  all  terms  in  the 
Programmer's  Manual  not  defined  elsewhere  in  the  document. 
A  list  of  applicable  documents,  including  cited  and  uncited 
references,  should  be  provided  in  Appendix  B.  Appendix  C 
should  provide  an  alphabetized  index  that  gives  the  page  on 
which  each  subject  contained  in  the  Programmer's  Manual  may 
be  found.  If  the  Programmer's  Manual  is  divided  into  more 
than  one  volume,  the  index  in  the  first  volume  should  be  the 
index  of  the  volumes.  The  index  in  each  of  the  remaining 
volumes  should  reference  only  those  subjects  within  that 
volume.  Appendix  D  should  provide  a  listing  of  model  test 
results  along  with  values  entered  into  the  model  that  pro- 
duced those  results.  Include  any  interim  model  outputs 
necessary  to  understand  the  final  outputs.  Provide  analyses 
of  model  results  as  necessary. 
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GUIDELINES   FOR   PREPARING  AN  ANALYST'S  MANUAL 


This  section  presents  a  recommended  organization  for  an 
Analyst's  Manual  and  describes  the  contents  of  each  section 
and  subsection  to  be  Included  In  that  manual.  An  Analyst's 
Manual  prepared  using  these  guidelines  will  enable  an 
analyst  to  understand  a  model's  functional  structure,  the 
algorithms  used  in  the  model,  and  techniques  employed  for 
model  verification  and  validation.  Figure  V-1  contains  a 
recommended  table  of  contents  for  an  Analyst's  Manual.  The 
sections  and  subsections  Included  cover  the  general  needs  of 
an  analyst  Interested  in  a  model.  In  documenting  a  particu- 
lar model,  however,  sections  and  subsections  may  be  added  to 
improve  clarity,  and  some  subsections  may  be  omitted  for 
s Imple   mode  Is  . 

1.  Introduction 

The  introduction  to  the  Analyst's  Manual  should  contain 
the  background  of  the  project,  the  purpose  of  the  model,  and 
an  overview  of  the  remaining  sections  In  the  manual.  A  com- 
mon introduction  may  be  used  for  all  the  manuals  prepared 
for  a  model,  but  the  specific  purpose  of  the  Analyst's  Manu- 
al  should  be   included   in  a   statement  of   the  form: 

"The  purpose  of  this  manual  is  to  provide  nonprogram- 
ming  analysts  of  (model  name)  with  the  details  of  the 
algorithms  used  in  the  model  and  the  techniques  em- 
ployed for  model  verification  and  validation." 


2.     Functional  Description  of  the  Model 


This  section  should  contain  a  well-structured  presenta- 
tion with  emphasis  on  the  functional  details  of  the  model. 
The  discussion  should  be  written  In  an  easy-to-understand 
manner  that,  whenever  possible,  avoids  the  use  of  highly 
specialized  terms.  The  section  should  be  divided  into  four 
subsections. 


2.1  Overview 


This  subsection  should  provide  a  functional  description 
of  the  model  in  sufficient  detail  for  an  analyst  to  under- 
stand the  salient  system  features  that  were  modeled.  Func- 
tional flow  charts  and  other  graphics  should  be  used  to 
enhance  the  narrative.  Include  a  statement  of  the  kind  of 
model  (i.e.,  discrete-event  model  that  simulates  jobs  enter- 
ing a  job  shop  at  arbitrary  points  in  time,  being  routed 
through  a  predetermined  sequence  of  production  centers,  and 
being  processed  at  the  production  centers)  and  the  degree  to 
which  the  model  portrays  the  real  world  system.  Figure  V-2 
is  an  example  of  modeled   system  highlights.      Included  should 
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be  the  set  of  model  responses  (output)  produced  by  a  given 
set  of  model  input  data.  Figure  V-3  provides  an  example  of 
the  types  of  model  input  and  output.  For  additional  details 
on  model  description,  the  analyst  should  be  directed  to  the 
appropriate   section  of   the   User's  Manual   for   this  model. 

2.2  Detailed  Methodology 

This  subsection  should  provide  the  functional  details 
for  analysts  to  understand  the  algorithms  and  equations  used 
in  the  model.  Well-known  mathematical  equations  (and  formu- 
lae) should  be  clearly  identified  and  references  should  be 
cited  for  their  derivation.  For  example,  in  a  job  shop  model 
the  queuing  discipline  simulated  for  each  production  center 
should  be  stated.  Include  the  derivation  for  extensions  of 
known  results  or  for  the  development  of  new  analytical  tech- 
niques. Special  complicating  details,  such  as  the  use  of 
precalculated  data  for  job  arrival  times  should  be  stated. 
The  description  must  be  detailed  enough  to  demonstrate  how 
the  model  uses  the  input  data  to  calculate  output  informa- 
tion. Functional  flow  charts  and  graphs  should  be  used  to 
enhance  the  narrative  descriptions  of  each  algorithm.  Fig- 
ure V-4  is  an  example  of  a  model  functional  flow  chart. 
This  section  should  include  a  subsection  for  each  major  al- 
gorithm  or   set   of  equations. 

2.3  Assumptions   and  Limitations 

This  subsection  should  list  all  model  assumptions  and 
all  factors  that  affect  or  limit  model  output  use.  The  fol- 
lowing  items   should   be    included   as  appropriate. 

2.3.1  Stochastic  Assumptions .  In  this  subsection,  itemize 
all  stochastic  assumptions  that  affect  model  output  accura- 
cy. For  example,  the  treatment  of  certain  random  variables 
in  a  simplistic  manner,  by  using  only  their  mean  values  and 
not  sampling  from  a  statistical  distribution,  should  be 
described  in  this  subsection.  Each  stochastic  item  should 
be  described  in  a  separate  subsection  (e.g.,  2.3.1.1, 
2.3.1.2). 

2.3.2  Magni  tude  Limltat  ions  .  This  subsection  should  include 
all   limitations   on   the    size   of   the   problems   the   model  can 
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address.  For  example,  the  current  dimensions  of  certain  ar- 
rays in  a  model  may  limit  the  number  of  activities  that  can 
be  represented  in  that  model.  Each  limitation  should  be 
described   in  a   separate  subsection. 

2.3.3  Critical  Values .  This  subsection  should  identify 
critical  input  data  values  to  which  model  outputs  are  sensi- 
tive. Many  elements  that  have  a  range  of  values  will  have 
one  value  that  is  particularly  significant  to  the  analyst. 
This  may  be  a  breakpoint,  a  minimum  stock  level,  or  a  criti- 
cal job  rate,  etc.  Each  critical  value  should  be  described 
in  a  separate   subsection  (e.g.,   2.3.3.1,    2.3.3.2,  etc.). 

2.4     Model  Flexibility 

This  subsection  should  address  the  capability  of  adapt- 
ing the  model  to  changing  requirements,  such  as  anticipated 
physical  system  operational  changes,  interacting  with  new  or 
improved  models,  and  planned  periodic  changes.  An  example 
of  a  flexible  design  is  one  that  facilitates  the  addition  of 
a  machine  failure  and  repair  subsystem  to  a  job  shop  model. 
Model  components  and  procedures  designed  to  be  flexible 
shall  be  clearly  identified.  Factors  that  affect  model 
flexibility  are  the  familiarity  of  the  analyst  with  the 
model,  the  model's  size,  its  complexity,  and  its  data  struc- 
tures.     Subsections   should   be   used   as  required. 

3.      Model    Input   and   Output  Data 

This  section  should  discuss  the  categories  of  input 
data  and  the  accuracy  of  model  output  data.  The  material 
contained  in  the  next  two  subsections  will  enable  the 
analyst  to  assure  the  existence  of  the  data  necessary  to  ex- 
ecute the  model  and  to  ascertain  the  accuracy  of  the  data 
generated   by   the  model. 

3  .  1     Input  Data 

Identify  all  categories  of  input  data  and  any  special 
analytical  techniques  required  to  obtain  those  data.  If  the 
sources  of  input  data  include  output  from  other  models,  pro- 
vide sufficient  details  to  enable  an  analyst  to  assess  the 
appropriateness  of  those  data  in  solving  his  problem.  For 
example,  if  the  arrival  rate  of  jobs  is  provided  by  a 
separate  model  of  the  order  handling  process,  the  analyst 
needs   to  determine   that   the   simulated   ordering  process 
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corresponds  to  the  one  existing  in  his  job  shop.  Details  on 
input  data  types  and  formats  should  be  reserved  for  the 
User's  Manual . 


3.2     Output  Data 


This  subsection  should  provide  the  analyst  with  a 
methodology  for  assessing  the  accuracy  of  model  output  data. 
Since  the  accuracy  of  the  output  values  will  be  judged  in 
relating  to  the  method  used  to  derive  them,  a  review  of  the 
algorithms  used  to  compute  those  output  values  may  be  neces- 
sary at  this  point.  Describe  in  detail  any  corrective  ac- 
tions to  be  taken  by  an  analyst  in  case  of  inaccurate  output 
values,  (i.e..  Should  the  analyst  contact  a  programmer  for  a 
program  change  or  have  a  user  modify  the  input  data  deck  to 
correct   the  problem?).      Subsections  may  be  used  as  required. 

4.     Model  Verification  and  Validation 

This  section  of  the  Analyst's  Manual  should  describe 
the  methodology  used  to  verify  and  validate  the  model. 
Model  verification  (sometimes  referred  to  as  software  vali- 
dation) is  concerned  with  the  compatibility  of  the  model's 
programmed  structure  to  the  analyst's  design  and  with  model 
debugging.  Model  validation  provides  the  analyst,  and  user, 
with  the  confidence  that  the  model  provides  a  good  represen- 
tation of   the  modeled  system. 


4.1     Verification  Techniques 


This  section  should  provide  an  analyst  with  concise 
procedures  by  which  the  model  was  verified.  Each  equation 
included  in  Section  2.2  of  the  Analyst's  Manual  should  be 
verified  and  cross-referenced  to  the  Programmer's  Manual  for 
this  model.      Include  all  other  verification  techniques  used. 


4.2     Validation  Considerations 


This  section  should  provide  an  analyst  with  the 
description  of  any  procedures  that  were  used  to  ensure  that 
the  model  is  an  "accurate"  abstraction  of  the  real  system. 
Any  methodology  used  to  determine  how  well  the  model 
represents  the  real  system  should  be  presented  in  this  sec- 
tion. While  complete  confidence  in  a  model  may  be  impossi- 
ble, a  good  validation  procedure  can  increase  the  amount  of 
confidence  an  analyst  has  in  a  model.  Figure  V-5  is  an  ex- 
ample of  a  graph  that  could  be  used  In  a  model  validation 
procedure . 
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APPENDICES 


Two  appendices  to  this  manual  should  be  provided  as  re- 
quired. Appendix  A  should  define  all  terms  In  the  Analyst's 
Manual  not  defined  elsewhere  In  the  document.  Appendix  B 
should  provide  a  list  of  applicable  documents  and  a  bibliog- 
raphy designed  for  use  by  system  analysts. 


-43- 


FREQUENCY  DISTRIBUTION  OF  JOB  TURNAROUND  TIMES 
COMPARISON  OF  ACTUAL  AND  SIMULATED  RESULTS 


ACTUAL 
SIMULATED 


TIME  (MINUTES) 


FIGURE  V-5 


-44- 


VI.      MODEL  SUMMARY 


It  is  suggested  that  documentation  for  each  model  in- 
clude the  information  outlined  below,  as  part  of  the  model 
documentation  package.  This  information  provides  general  in- 
formation about  the  model  and  facilitates  its  possible  use 
by  others. 

A.  Basic  Description. 

1.  Name  or   title   of  Model. 

2 .  De ve lope  r ( s  )  . 

3.  Agency  or  company. 

4.  Sponsor;   purpose   or  objective  of  sponsor. 

5.  When  developed? 

6.  Where   developed?  . 

7.  Development   time   and  cost? 

8.  Developed   separately  or  as  part  of  larger 
study? 

B.  Subject  Matter  of  Model. 

1.  Major  purpose   of  model. 

2 .  Scope   of  model . 

3.  State  basic  description  or   theory  underlying 
the   model . 

4.  State   specific   disc ipl ine ( s)   required   for  model 
use  ,    if  required  . 

5.  How  does  model   differ   from  other  similar 
mode  Is? 


Modeling  Technique. 


(e.g. 

)? 

mode  1 ? 


Describe   type   of  model. 
Does  model  use  any  standard  packages 
linear   programming,   statistical,  etc 
Was  the  model  developed   from  another 
If   yes,   describe  process. 
Is   its   structure   clear?     Its  variables? 
Describe  data   requirements  of  model. 
Does  model   receive  any  data   from  other  models? 
What  constraints  are  affecting   the  model? 


Computer  Aspects  of  Model. 


1.  In  what  computer  language   is   the  model  written? 

2.  What  machine(s)    is   it   programmed  for? 

3.  How  much   time  does   it   take   to  run? 

4.  Size   of  model   (lines  of  code,   core   to  run  etc.) 

5.  How  many  parameters  does  model  require? 
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E.     Validation  of  model. 


1.  Has  model   been  validated?  How? 

2.  Has  model   been  documented?     How  well? 

3.  Has  model  been  critiqued  or  appraised?     By  whom? 
At  what  point? 

4.  Has   there   been  a   sensitivity  analysis  performed 
on  the  model?     By  whom? 

5.  Can  the  model  by  used   from  current  documentation? 
Has   it  been  used? 


F.      Model  Use. 


1.  If   asked,   how  would  you  demonstrate   the  utility 
of   the  model?     Have   you  demonstrated  it? 

2.  With  whom  should  one   get   in  touch  to  discuss 
use   of   the  model? 

3.  How  much  would   it  cost   to   transfer   the  model? 

4.  Are   the  model   relationships  or  parameters  easy 
to  use   for  the  user? 

5.  Have  there  been  any  papers  given  or  written  on 
the  model?     Cite  references. 

6<      Is   the   output   of   the  model   special  or   is  it 
designed   for  a  general  audience? 
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VII.  SUMMARY 


The  guidelines  on  the  previous  pages   should  assist  in 
the  preparation  of  documentation  for  computer  models.  Such 
documentation  is  primarily  a  tool  for  human  communication. 
Varying  information  needs  of  different   types   of   readers  such 
as  managers,   model  users,   programmers,   and  analysts  are 
accommodated.     While  managers  are  often  in  need  of  a  broad 
spectrum  of   general   information  required   for  decisionmaking, 
users  are  primarily   interested   in  practical  aspects   of  the 
model  and  its  application   to   the  user's   specific  problems. 
Programmers   require  technical  details  which  are  needed  for 
maintenance  and  modification  of  the  model,   while  analysts  are 
interested  in  the  processing  aspects  and  the  underlying 
analytical  methods  and  algorithms.      By  providing  sections 
which  have  been  specifically  tailored  to  the  viewpoints  of 
diverse  readers,   human  communications  are  enhanced  and 
differing  information  needs  are  satisfied. 

Users  of  these  guidelines  are  requested  to  provide  feed- 
back on  their  use  of   this   document   to   the  authors.      It  would 
be  of  interest  how  well  the  document  has   served   the  user's 
needs,   what  parts  have  been  especially  useful,  what  parts  were 
not  used  and  why,   and  what   changes  or  additions  are  suggested 
by  the  readers  and  users   of  the  document.      Such  comments  would 
be  used  in  future  revisions.     Use  of   this  user  experience 
would  be  of  great  value   to  the  Federal  Modeling  Community  to 
which  this  paper   is  addressed. 
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NBS  TECHNICAL  PUBLICATIONS 


PERIODICALS 

JOURNAL  OF  RESEARCH— The  Journal  of  Research  of  the 
National  Bureau  of  Standards  reports  NBS  research  and  develop- 
ment in  those  disciplines  of  the  physical  and  engineering  sciences  in 
which  the  Bureau  is  active.  These  include  physics,  chemistry, 
engineering,  mathematics,  and  computer  sciences.  Papers  cover  a 
broad  range  of  subjects,  with  major  emphasis  on  measurement 
methodology  and  the  basic  technology  underlying  standardization. 
Also  included  from  time  to  time  are  survey  articles  on  topics 
closely  related  to  the  Bureau's  technical  and  scientific  programs. 
As  a  special  service  to  subscribers  each  issue  contains  complete 
citations  to  all  recent  Bureau  publications  in  both  NBS  and  non- 
NBS  media.  Issued  six  times  a  year.  Annual  subscription:  domestic 
S13;  foreign  $16.25.  Single  copy,  $3  domestic;  $3.75  foreign. 

NOTE:  The  Journal  was  formerly  published  in  two  sections:  Sec- 
tion A  "Physics  and  Chemistry"  and  Section  B  "Mathematical 
Sciences." 

DIMENSIONS/NBS— This  monthly  magazine  is  published  to  in- 
form scientists,  engineers,  business  and  industry  leaders,  teachers, 
students,  and  consumers  of  the  latest  advances  in  science  and 
technology,  with  primary  emphasis  on  work  at  NBS.  The  magazine 
highlights  and  reviews  such  issues  as  energy  research,  fire  protec- 
tion, building  technology,  metric  conversion,  pollution  abatement, 
health  and  safety,  and  consumer  product  performance.  In  addi- 
tion, it  reports  the  results  of  Bureau  programs  in  measurement 
standards  and  techniques,  properties  of  matter  and  materials, 
engineering  standards  and  services,  instrumentation,  and 
automatic  data  processing.  Annual  subscription:  domestic  $11; 
:   foreign  $13.75. 

!  NONPERIODICALS 

Monographs — Major  contributions  to  the  technical  literature  on 
various  subjects  related  to  the  Bureau's  scientific  and  technical  ac- 
tivities. 

Handbooks — Recommended  codes  of  engineering  and  industrial 
'  practice  (including  safety  codes)  developed  in  cooperation  with  in- 
I  terested  industries,  professional  organizations,  and  regulatory 

bodies. 

Special  Publications — Include  proceedings  of  conferences  spon- 
t  sored  by  NBS,  NBS  annual  reports,  and  other  special  publications 
appropriate  to  this  grouping  such  as  wall  charts,  pocket  cards,  and 
bibliographies. 

Applied  Mathematics  Series — Mathematical  tables,  manuals,  and 
studies  of  special  interest  to  physicists,  engineers,  chemists, 

I  biologists,  mathematicians,  computer  programmers,  and  others 

'  engaged  in  scientific  and  technical  work. 
National  Standard  Reference  Data  Series — Provides  quantitative 

(i  data  on  the  physical  and  chemical  properties  of  materials,  com- 

I  .piled  from  the  world's  literature  and  critically  evaluated. 

|l   Developed  under  a  worldwide  program  coordinated  by  NBS  under 

jl  the  authority  of  the  National  Standard  Data  Act  (Public  Law 

I  90-396). 


NOTE:  The  principal  publication  outlet  for  the  foregoing  data  is 
the  Journal  of  Physical  and  Chemical  Reference  Data  (JPCRD) 
published  quarterly  for  NBS  by  the  American  Chemical  Society 
(ACS)  and  the  American  Institute  of  Physics  (AIP).  Subscriptions, 
reprints,  and  supplements  available  from  ACS,  1 155  Sixteenth  St., 
NW,  Washington,  DC  20056. 

Building  Science  Series — Disseminates  technical  information 
developed  at  the  Bureau  on  building  materials,  components, 
systems,  and  whole  structures.  The  series  presents  research  results, 
test  methods,  and  performance  criteria  related  to  the  structural  and 
environmental  functions  and  the  durability  and  safety  charac- 
eristics  of  building  elements  and  systems. 

Technical  Notes — Studies  or  reports  which  are  complete  in  them- 
selves but  restrictive  in  their  treatment  of  a  subject.  Analogous  to 
monographs  but  not  so  comprehensive  in  scope  or  definitive  in 
treatment  of  the  subject  area.  Often  serve  as  a  vehicle  for  final 
reports  of  work  performed  at  NBS  under  the  sponsorship  of  other 
government  agencies. 

Voluntary  Product  Standards — Developed  under  procedures 
published  by  the  Department  of  Commerce  in  Part  10,  Title  15,  of 
the  Code  of  Federal  Regulations.  The  standards  establish 
nationally  recognized  requirements  for  products,  and  provide  all 
concerned  interests  with  a  basis  for  common  understanding  of  the 
characteristics  of  the  products.  NBS  administers  this  program  as  a 
supplement  to  the  activities  of  the  private  sector  standardizing 
organizations. 

Consumer  Information  Series — Practical  information,  based  on 
NBS  research  and  experience,  covering  areas  of  interest  to  the  con- 
sumer. Easily  understandable  language  and  illustrations  provide 
useful  background  knowledge  for  shopping  in  today's  tech- 
nological marketplace. 

Order  the  above  NBS  publications  from:  Superintendent  of  Docu- 
ments, Government  Printing  Office,  Washington,  DC  20402. 
Order  the  following  NBS  publications — F/PS  and  NBSIR's—from 
the  National  Technical  Information  Services,  Springfield,  VA  22161. 

Federal  Information  Processing  Standards  Publications  (FIPS 
PUB) — Publications  in  this  series  collectively  constitute  the 
Federal  Information  Processing  Standards  Register.  The  Register 
serves  as  the  official  source  of  information  in  the  Federal  Govern- 
ment regarding  standards  issued  by  NBS  pursuant  to  the  Federal 
Property  and  Administrative  Services  Act  of  1949  as  amendec 
Public  Law  89-306  (79  Stat.  1127),  and  as  implemented  by  E> 
ecutive  Order  11717(38  FR  12315,  dated  May  11,  1973)  and  Par 
of  Title  15  CFR  (Code  of  Federal  Regulations). 

NBS  Interagency  Reports  (NBSIR)— A  special  series  of  interim  or 
final  reports  on  work  performed  by  NBS  for  outside  sponsors 
(both  government  and  non-government).  In  general,  initial  dis- 
tribution is  handled  by  the  sponsor;  public  distribution  is  by  the 
National  Technical  Information  Services,  Springfield,  VA  22161, 
in  paper  copy  or  microfiche  form. 
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